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The internal resources an agency devotes to its own 
programmatic /review and assessment can be most effectively related to 
the agency • y administrative decision-making if the agency is 
self-evaluating and uses Havelock's model of linkage between formal 
research dissemination and utilization as a framework for the 
integration of program evaluation andr administrative decision-making. 
The aspects of the Havelock model discussed here are linkage, which 
is the degree of connection among groups; structure, which involves 
tlie degree of systematic organization and coordination of elements; 
openness, which is the readiness to give and receive information; 
capacity, which is a summary concept tying together a number of 
variables that involve having the resources and competence to perform 
satisfactorily; reward, which has to do with positive reinforcement; 
proximity, which involves nearness in the physical sense of time and 
place as well as in the psychological sense of familiarity^ and 
similarity; and synergy, which includes the number, variety, 
frequency, and persistence of forces t-hat are used to communicate 
evaluative findings. (Author/IRT) 
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How can the internal resources an agency devotes to its own prograroroatic 
review and assessment be most effectively related to the agency's administrative 
decision making? This will be the primary question t^is paper will address. But i 
before this, however, there is a more basic question: How should organizations 
(in this case, human service agencies), view their own purpose? The answer to 
this question assumed In this paper shall be that proposed by WildaVsky (1972) — 
they should be self -evaluating organizations. This self --evaluation is a 
continuous monitoring process of ^he agencies' own activities assiessing whether 
goals are met and even whether these goals are still relevant. Evaluation data 
suggesting inefficiencies is taken seriously by administrative decision mak- 
ers ,/becomlng the basis for program development and change. Such an organization 
would be a paragon of efficiency and effectiveness. No doubt, full development of 
such an organization Is unobtainable. Yet it remains the Ideal and the premise for 
program evaluation and the basis for its integration vithin organization. The dis- 
cussions below suggest ways to further the development of human service agencies 
toward becoming self-evaluating organizations* 

Although much has been written about formal research dissemination and util*- 
ization (e.g., Havelock, 1971, Rogers and Shoemaker, 1971; DHEW Series, 1972), very 
little has been said about the utilization of internal evaluative research findings 
within an agency (Davis^^and Salasln, 1975). Consequently, there are not numerous 
models to. choose. However, there are a number^ of models proposed for formal research 
dissemination and utilization. Of these, Ronald Havelock' s paradigm provides a 
framework around which the most important aspects of the integration by program 
evaluation and administrative decision making can be discussed and guidelines drawn. 

Havelock sees dissemination and utilization as an act o£ communication 
between a research resource system and a user system. A resource system includes 
the body of knowledge cf some area of research plus the persons informed of this 
knowledge and any other resource which could be used to impart this information. 



A user system Is some applied area In which this knowledge, if adopted in the 
form of innovation, might result in improved performance. Integration is created 
through two-way Interaction processes which connect resource and user systems. 
Both systems function more successfully when there is ample communication between 
them involving an exchange of two-way messages which continuously stimulate 
each other's problem-solving behavior. Seven factors are required to describe 
how both systems are inter-connected and to account for degree of successful 
utilization for research results. 

The administration and practitioner parts of human service agencies would 
be comparable to Havelock's user system and agency's program evaluation capa- 
bilities is commensurate to the resource system. Program evaluation should not' 
be viewed as just the persons and immediate resources available, but the 
potential range of consultants, prior research findings, and information sys- 
tems which could be brought to bear. 

Linkage 

The first and most important concept is "linkage." Indeed, the other six 
factors can be viewed as additional ways to explain and enhance it. Linkage is 
the degree of conm tiveness that exists among groups. Between resource and 
user systems, it can be measured by the number, variety, and mutuality of con- 
tacts and by the degree to which they share collaborative relationships. The 
more linkages and the stronger they are, the more effective contacts will be and 
the greater the impact one system will have on the other. Information will be 
more easily exchanged and influential. Research in a high linkage situation 
will develop to more closely meet user needs and will be more readily accepted. 

• Linkage is seen as the key concept by Havelock because he and the other 
reviewers of research dissemination and utilization have found that a gap exists 
between research findings and practice. Each arises from a system independent 
of the other. Researchers tend to write for other researchers and are read only 
by them. Practitioners are not knowledgeable about research and modify their 
programs based on conversations with other practitioners. As a result, 
researchers tend to do research of little interest to practitioners and 



practitioners continue to run programs with disproved or unproved methods. 
Linkage, however, between these two has been shown to improve relevancy of 
research and produce programs incorporating methods of proven merit. 

There is such a gap within human service organizations. On one side is 
the administrators and practitioners of the agency involved in the day-to- 
day management and delivery of service. On the. other is a much smaller number 
of persons who are ^responsible for evaluation, the linkage between them is 
typically weak and ill-defined. Linkage involves both formal and informal 
.contar.t. Without underestimating the power of informal communication and 
relationships, currently the greatest deficiency in human service agencies is 
the lack of well-defined formal linkage between evaluation and administration. 

All too often the processes of evaluation and administration go their 
'^independent waysi not interacting with one another. Missing is the formal 
link, the person or persons who overlaps in responsibility for evaluation and 
administration and understands both. This is more than merely an assignment of 
responsibility Shown in an organizational chart. The critical element is the 
real. and knowledgeable participation in both processes. 

Instead of real linkage there commonly is merely the appearance -ot it. ' 
To illustrate this» consider an example of pseudo-linkage in the situation, 
where an evaluator reports to an administrator. The administrator who has 
"had no training in evaluation or research is uneasy by the prospect of supervising 
the evaluator^ who he feels may reveal his ignorance. The ^valuator . coming from 
a i;esearch background is concerned wit|i reporting tp^an administrator who does 
not understand research and may be unsympathetic. Each is unfamiliar with 
their respective roles. A weak relationship based on mutual anxiety and lack of 
understanding develops between them. The evaluator is left pretty much to do 
evaluative research of hJ ^ own choosing rarely actually meeting with the 
administrator, but generating written reports. The administrator sees.the reports, 
although he has a limited understanding of them, as tangible evidence of the 
productivity of the evaluator and his own successful supervision. Yet, he only 
makes limited and sporadic use of findings, if at %L1; Administrative decision 
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making and possible Information the evaluator might supply are never* or only 
Infrequently^ discussed by them. The relationship never evolves to a point " 
where the admli^strator defines his or the agency's decision-making Infor- 
mation needs to which evaluator responds. Administration and evaluation are 
left as separate prooa^es. ^> 

Conceptually, there are tWo models for a formal role linking evaluation 
and administration at top level of management within an organization: 1) the 
admlnlfltrator-evaluator ; 2) the evaluator-admlnlstrator . The administrator- 
evaluator Is an administrator who has management responsibility for evaluation 
among oi:her responsibilities. He Is not typically a professional evaluator. 
The^professlonal evaluator who directly conducts and supervises evaluation » 
repoi^..;to him. Although not a fully trained evaludtor, the admlnlstrator- 
evaluator either has obtained some formal training or gained an understanding 
of evaluation through experience. He knows Its general methods of operations, 
strengths, and .limitations. By his administrative position, the adminlstrator- 
evaluator observes and participates In. decision making. Consequently he 
possesses an understanding of these agencies' declslon-malclng Information needs. 
He sees evaluation as a tool and basic component >of the agency whose purpose 
Is to supply the needed Information for administrative decision making. His 
function is to translate these needs to the evaluator, to monitor the degree 
of 'BMCcess to which the evaluator meets these needs, and input the information 
into administrative decision making. 

The second model of effective linkage is the evaluator-admlnlstrator. He 
is a" formally research-trained evaluator who directs'' and conducts the agency's 
evaluation activities. In addition,"^ he regularly participates in the agency's 
high level decision making. He attends administrative meetings where manage- 
ment concerns are discussed and is^ree to communicate Information. He frequently 
uses such occasions as an opportunity to convey evaluative information, both 
written and oral. He also takes an active part in deliberations expressing his 
own opinions and is seen as sharing in the ultimate decision. He shares in 
both evaluation and administration. By being directly involved in the decision- 
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making process, he knows the agency's administrative Information needs. This 
permits him to plan both short and long range evaluative activities sb that useful 
Information will be available. The evaluator<-admlnlstrator Is responsible for 
the full range of evaluation steps; from planning st,udles» to data collection, 
to synthesizing, and finally to reporting to administrative decision makers. 

The admlnlstrator-evaluatpr artd evaluator-admlnlstrator appear as promising 
formal linking roles. The former is- probably closer tO:<. typical current practice. 
The training of administrators in program evaluvtlon is a necessary step to mak- 
ing this model widespread. The training need should not be to turn the admlnls- 
trator inJ:o an evaluator, but to educate him to be able to administer evaluation. 
The only additional responsibility is for him to function as a major source of 
evaluative information to other administrator decision makers. The evaluator- 
admlnistrator role is more controversial and necessitates a change in the perception 
of the evaluator 's function. The active participation in adihlnJLstration as a 
stated part of his responsibilities runs counter to the training of researchers. 
Yet, there is an inevitable pull on evaluators to become Involved tn adminis- 
tration. Rather than reject such activism, it might bife better to structure it 
and ensure that evaluation is successfully linked to administration. 

Linkage should not be viewed as just occurring at the top of the agency 
structure. Evaluator and administrators must be formally linked at the various 
administrative levels. At lower levels, evaluators should have a formal consulta- 
tive role with managers. Moreover, as mentioned above, effective linkage requires 
close informal contact and relationships.^ However,* the key to first establishing 
effective formal linkage and from this will come viable Informal communication. 

Structure — 

"Structure" Involves the degree of systematic organization and coordination 
of elements. In order for evaluative information to be successfully Incorporated 
into administrative decision making, there must be an organized fraitiework. A 
structure with a rational sequence by steps, and a compartmentalized and coordln-^ 
ated division oX labor within .the agenc}^ There must be structure for the agency 
as a whole and within and between the administrative and evaluative components. 
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There are four ways structure Is directly Important for evaluation. Flrsti 
evaluation to be effective needs to have Its own division of labor and coordln~ 
atlon of effort. ^Whether conducted by one person or a large staff, evaluative 
activities should be organized Into a system. Secondly, evaluation should 
have a structured and coherent view of the rest of th^^ agency. It should be 
able to understand the various levela atid sub-cojnponents and how they are 
Interrelated. It should know the needs of the clients and the organization 
goals* It should have a good view and understanding of the agency's decision* 
making process. Thirdly, evaluation should be able to plan how Information It 
has collected will be Inputed In a structured sequence Into the decision-making 
process. Like the development and operation of evaluative studies, data Input 
9hpuld be a planned, structured activity — not a hlt-or-mlss afterthought. 
Fourth, evaluation must be structured within the agency so that It typically is 
not adminlstrati\^ely. under those who are directly responsible for the management 
of th\e program b^ilng evaluated. The formal linkage point between evaluation and 
administration should be^^t a\hlgh' level. Although evaluation must be Integrated 
into the agency, it needs to maintain as much Internal autonomy as possible. 
Evaluation ^st:aff should work closely with program arid administrative personnel, 
but be separate and Independent from them. Evaluation staff should have a 
commitment to the agency's goals, but not to the methods chosen to reach them. 
This is the major difference between them and others within the agericy. 

Structure is of ' critical Importance to administrative-practitioner parts of 
the agency if .evaluation is going to have influence. As Horst et al (1974) have 
pointed out, if the program Itself is not well organized it cannot Incorporate 
evaluative information and modify programs. Administration must have effective 
structured problem solving and decision making process, or evaluative information 
regardless of its potential value will be of limited use. Moreover, just as 
evaluation shbuld be organized to ^s^rid information, administration should be 
organized to receive. There should be built-in times and places where admini- 
stration expects and requests evaluative data. 
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Openness 

The third factor ^Is **openness," the readiness to give and receive 
information. It requires a willingness and readiness for the user system to 
accept help as well as a willingness and readiness of the resource to listen 
to user needs and to give help. Openness is based upon agencies acceptance 
of the ethic of the self-evaluating organization that change is desirable and 
possible. As long as a human service agency sees its purpose is to deliver 
fixed types of seemingly inherently valid services, openness will not occur.* 
Information for such, an agency will be viewed, at b<^st as not needed, and at 
worst as a threat to its exis^encl^'. However, without openness between evalu- 
ation and administration human service ^agencies will continue to deliver 
unproven or disproven types of services of limited usefulness. 

For evaluators, openness means being willing to collect information that 
will be of use to administrators. This in turn requires the evaluator to listen 
to administrative needs and develop responses tailored to meet them« A primary 
evaluation objective^ then, is the providing of information witt> utility for 
administrative decision-making. As a consequence, the validity of findings 
may be lessened; io a point, evaluators must be willing to run this risk. 
Obviously, if validity is too greaitly reduced, the information will be meaning- 
less. A part of the prpfessional skill of the evaluator is to Judge ^the relative 
trade-offs between validity and utility needs and decide the proper balance. 

For administratorff, openness^' means being willing to expose areas where .. 

they are uncertain or believe there are deficiencies. It should be an active 

proce63-iDf requesting and seeking out information rather than passively waiting. 
^-^.^ • • ■ . ' . ■ ' 

to be asked and provided for. As part of a self-evaluating agency,, they are ^ 

not held Accountable to run perfect programs, but. to be actively attempting to 

improve them. 

Capacity 

"Capacity" is a summary concept tying together a. number of intercorrelat^d 
variables such as "wealth," "power," "status," "education," "intelligence," \. 
and "sophistication." It involves having the resources and competence needed 
to perform satisfactorily. Wirthin human service agencies, both evaluation and 
administration must have the capacity to do their Jobs in order for evaluation^ 
findings to be utilized. R 



The concept of evaluation information capacity was important e.nough to be 
discussed separately above. In addition, evaluation must have sufficient 'top*- 
ital and ability to summon and invest the resources required to collect valid^' 
and useful information';/ Once Infprmation is collected, it must have the skill 
based on expjsrience of imparting the data, to administration in ways which 
heighten acceptability. Finally, it neeJs adequate power and prestige to 
ensure that this information will be attended to. Capacity, however, is rela** 
tive and should be geared to the agency. Attkisson et al (1974) report that 
the type of persons who have functional role of the evaluator in different 
agencies range, from statistical clerks to professionally trained researchers 
who participate in high level administrative decision making. These types of 
persons reflect the capacity of these agencies to conduct evaluation. For 
small agencies, a clerk may be sufificlent; however, for larger, more complex 
ones, greater capacity is required. Even in the small agency, an administrator 
will need to direct and supervise the clerk~in effect, assuming part of the 
evaluation function; hence even small agencies need to have a relatively large 
commitment for meaningful evaluation. The over-all resources and competence 
of the agency should be matched with comparable evaluative capacity. 

The combination of administration structure and capacity is that Horst'et al 
(1974) call maoagement. Without good management, they feel that evaluation, no 
matter how good by itself, will not .change or improve ^agencies. Clearly, in 
addition to good administrative structure, agencies must have the capacity to 
run programs well before they can incorporate evaluative information. Like 
evaluators, administrators must have the ability to assemble and invest program 
resources. Moreover, they need self-confidence as well as administrative skill 
and sophistication. Cood administration is the basis both of-^ good programs 
and effective evaluation. 

Reward 

The fifth factor is "reward." By this, Havelock means the frequency, 
immediacy, amount, and multiplicity of planning and structuring of positive 
reinforcements for resource and user systems. In order for evaluation to be 
influential in human service agencies, a system must be created such that 



evoluators arc rewarded for good evaluation and administrators arc reinforced 
for Incorporating evaluative findings into their decision making. The simple 
Fact Is that now neither evaluators nor administrators are regularly or fre- 
quently rewarded for these activities. Often, the evaluator-*"-even if only pa'rt 
of the findings are critical of current functioning — is ignored or even becomes 
a target of attack. . The evaluation design with its inherent methodological 
limitations may be compared to narrow research standards and is concluded to be 
Inadequate. Frequently the study is then disregarded and has no effect. Even 
if findings may have some^ eventual effect on program changes and improvement^, 
beyond the satisfaction of seeing this the evaluation niay be a personally 
negative experience for the evaluator. For the administrator, the evaluation 
.can be equally unsatisfactory. Although expecting confirmation of this program' 
worthy a frequent result is an' evaluation which ±B negative, a^ least about some 
nidjor 'part of progi^api operation, and perhaps even shows no over-*all demonstrated 
effectiveness. Such findings might Jeopardize the program's funding. He may 
feel his own professional ability is under attack, and indeed he may find 
himself being blamed by others, including his own staff, for the poor results 
rather than their accepting programmatic failure. 

The lack of reward for participation in evaluation has a stifling effect. 
Campbell (1969) argues that new human service programs should be conceptualized 
as social experiments ahd not proposed as final solutions. They should rather 
be viewed as attempts, aJrthough well thought out and t>i^omlslng ones, to meet 
certain social problems. It is true that If evaluation were. viewed so, adverse 
effects would be diminished. In light of the negative evaluative findings, 
admin latra tors of human service agencies may begin to hold a more modest opinion 
about their programs' degree of success, stating them conditionally. Then, too, 
evaluators must modify the type of evaluations they do. Too frequently, evalu** 
ators take the pre-evaluatlon rhetoric about programs* effectiveness literally 
and design studies aimed to assess lofty objectives^ ensuring that more modest 
achievement will be assessed as failure. While for summatlve evaluation stated 
goals have to be taken literally, most evaluation should.be formative, intended 
for program Improvement, determining success to be that which extends beyond 
current practice. With adjustment in administrative expectation and commensurary 
modification eva'' : ^lon strategy,' meaningful assessment of human service can be 
done that can often reward both. . ^ ^ 
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Proxlml ty ^ ^ 

'Vproxlmlty" Involves t1ye nearness in the physical sense .of time and plocc 
AS Well as in the psychological sense of familiarity and similarity between _ 
Ciset arid,.resource systems. The closer the proximity between evaluators and 
administrators, the more likely effective linkage between them, an<f in turn the 
'greater, utilization of evaluat^v6 information. Evaluatorg ought not be distant 
figures having their own "ivory towex" within the agency. They should be^ 
ideated close to administrators and Rractitidners, seering them frequently— 
formally and informally • The dvaluatdr s'hould be able to speak- in the terms 
and slang of the agency as an insider erttf be seen as "one of us/' although with 

a different function. ■ ^ 

^ \^ ■ ^ 

There is also another meaning to "prox^mity'^ the proximity of evaluative 
information to the administrative decision-making process. Again, proximity has 
both a physical and psychological sens^. Evaluation information should be close 
±n time with' administrative decision, making. If ^fctvls given long h^fore the 
decision* br after, It will be lof limited or no use . v Time and pldce are- critical 
to administrative decision makers. Evaluators must further modify evaluation 
designs often at the cost of validity in order that 8ome\lnforination will be 
there when thfe decisionals to be made. Psychologically , )Bva^u^tive \information 
should be close to users' needd.. It should be seen by ^dminl^tratotsras related 
to their decision making. To the exteht that finding need-interpretati^^o^ and 
translation to administrators before their usefulness jcan be undfera^dd^i |:he more 
likely they will be disregarded. ^ ^^\^^^^ 
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The final factor, Havelock calls "synergy." It includes the number, V4W)a|y^,^ 
frequency, and persistence of forces that are used to communicate evaluative^^^^ 
flndingp. His point is that successful research utilization required iredundaiiiy^^^^^^^^ 
in communication. Evaluative information, like other information, must be V^^^^,';^^ 
repeated over and over again until it gets attended to and absorbed. Moreover, 
this redundancy should be organized in a coherent fashion presenting the message 
in a number and variety of forms. 
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The concept of synerfty asks the e valuator again to modify what he was 
taught In his research training. Researchers are educated that end of a suc- 
cessful study Is 4 written report, hopefully published In a Journal. Evaluators 
tooy typically see\the end of the evaluation as a final writteri report. However 
if the evaluator isX^erious about his activities having an impact on the agency » 
then the classic eval^uation report is no more than one of a series of ways by 
which he will communicate the Information. Evaluation activity does not end ) 
with a written report, \but moves into the internal dissemination stage* The 
evaluatlor should develop a dissemination strategy to present the information 
in a variety of ways. The evaluator should fully utilize the linkage with 
administration, presenting information in meetings, both formal and informal,' 
using summary abstracts as well as complete '.written reports, giving talks, 
making slide presenfations, etc. Mu^h evaluative information continues to be 
relevant long after the data collection and. initial wrlt^up of results. Like 
old "wine, such Information should be brought up from the written report cellar 
and served from time to time. Administrators , will come to see the evaluator as 
a repository of useful information and will actually seek out input into their 
decision making. Thus, synergy will become an active process both for evaluators 
and the administrators. 
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